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Executive Summary 


In a budget note in HB 5016 (2015), the Oregon Legislature directed the Oregon Department of 
Education (ODE) to report back to the Legislature prior to March 1, 2016 on its efforts to provide 
direction, and if necessary, training to staff from school districts and education service districts for 
reporting graduation data. This note also asked Department to initiate direct contact with school 
districts that report a graduation rate below the state average to verify the accuracy of the data 
before it is released publicly. The following report provides details on ODE’s ongoing training 
efforts, as well as its outreach to school districts with low graduation rates. 

The Oregon Department of Education (ODE) produces graduation and dropout reports annually, 
as part of the accountability and reporting process. Beginning in 2008-09, these reports included the 
cohort graduation rate. At that time, report validation was completed in March of the following 
school year, and reports were released in April. In 2011-12, the timeline was moved up to 
accommodate federal reporting deadlines, and validation is now completed in early January of each 
year, with the reports released at the end of January, and Oregon’s six federally required graduation 
and dropout reports filed shortly thereafter. 

Since the first calculation of these rates in 2010, ODE staff from Assessment & Accountability and 
Information Technology offices built significant improvements into the process, in order to assist 
schools and districts with submitting correct data, and more efficiently allocate staff time to working 
on data quality. 

The current process includes data quality checks throughout the school year, up to 10 recorded and 
live webinar trainings for data submitters and other interested school and school district staff 
members, and a number of extensive policy, technical, and data collection manuals. It also involves 
the development, maintenance, and continuous improvement of programming to identify potential 
errors and contact data submitters for review (through email and through our established audit 
review process). 

As a result of the tremendous efforts of school and school district staff to validate their data, four- 
year cohort graduation rates typically increase between 3 and 5 percentage points from the 
preliminary rate to the final, published one. Data quality improvements in graduation and dropout 
reporting also yield benefits in other areas, as this data helps ODE to enforce state school funding 
rules, validate other required annual reports, and respond to data requests from legislators, policy 
and advocacy groups, and other stakeholders, throughout the year. 

Examples of manuals, trainings, contacts, and record-level audit review can be found throughout 
this report. This report can be found online at http://www.ode.state.or.us/search/page/?id=2644 . 



Cohort Graduation 
Training and Validation Report 

(House Bill 5016) 


Background and History 

High School graduation rates are key indicators of accountability for high schools and school 
districts in Oregon. Beginning with the 2008-09 school year, the Oregon Department of 
Education (ODE) implemented the cohort method of calculating graduation rates. The cohort 
graduation rate is published annually, with the most recent report published on January 28, 
2016. In addition to graduation, these reports include other outcome categories for students in 
the cohort, such as GEDs, continuing enrollment and non-completion. These reports are also 
disaggregated by student groups such as race/ethnicity and gender and are reported at both the 
school and district level. 

A cohort graduation rate follows the enrollment of students who are first-time high school 
students in a particular year and determines the percentage that graduate within a given time 
frame, such as four years. For example, a four-year cohort graduation rate for the first-time 
ninth graders in the 2010-11 school year would represent the percentage who graduated by the 
end of the 2013-14 school year. An extended rate, allowing one additional year for completion, 
is also tracked and reported as the 5-year Cohort Graduation Rates. 

Each cohort begins with the set of first-time 9th graders that enroll in the fall of each year. 
Cohorts are continually adjusted by adding in those students who transfer into a school or 
district during the period being measured, and by removing those students who transfer out of 
the school or district. Dropouts are not removed from a school or district’s cohort. At the end 
of the four- or five-year timeframe ODE calculates the graduation rate as the number of 
students earning a high school diploma divided by the number of students in the adjusted 
cohort. 

The calculation of cohort graduation rates is a complex one: it must follow a student’s 
enrollment for four or five years in order to determine the final outcome for each student. 

This is fairly straightforward for students who remain at a single school, but becomes 
increasingly complex as students transfer from school to school and district to district. In 
addition, the calculation must record the students’ race/ethnicity, and special education, 
economically disadvantaged, English learner and migrant status during high school. 

Because of the complex nature of the calculation it is vitally important that districts are 
provided with training, assistance, and the data they need in order to report student outcomes 
and to validate data on their cohorts. This document describes the process and methodologies 
that ODE uses to help ensure that the cohort graduation data are of the highest possible 
quality. 

Manuals, Trainings, and Documentation 

ODE staff conduct multiple video trainings throughout the school year to assist districts in 
submitting and validating their data. These trainings are conducted via interactive webinars, and 



recordings of the trainings are preserved on ODE’s website for district staff that missed the training 
or would like to review it. 

• Four trainings are conducted each year on the submission of data to Cumulative Average 
Daily Membership (ADM) collections. (For examples of training presentation, see First 
Period. Annual ; for sample recordings, see First Period . Annual) . 

• Additional Cumulative ADM trainings are added when complex new requirements are 
introduced; for example, in 2015-16, there was a fifth training on how to report full-day and 
half-day kindergarteners. 

• A training is conducted in early June that covers the use of the ADI validation application 
Glides: Recording) 

• A training in mid-October covers the Exit Adjustment window and the basics of the 
Graduation and Dropout Rate calculations (Slides; Recording) 

• A training in late October covers the details of the Graduation and Dropout Rate 
calculations and validation process Glides: Recording! 

• Additional video trainings are conducted by staff from other offices on the supplemental 
sources of data for graduation and dropout calculations, such as the ESEA Title III: Limited 
English Proficiency /English Learners collection, the December and June Special Education 
Child Counts and Exits, and the Title X: Homeless collection. 

In addition to the trainings. Accountability & Reporting staff produce multiple manuals and 
guidance documents for districts to reference. These include: 

• The Cumulative ADM Manual (127 pages) - Policy guidance, technical instructions, mles, 
and interpretations for submitting data to the Cumulative ADM collections. 

• The Cumulative ADM Audits and Reviews Manual (35 pages) - Guidance on interpreting 
and resolving audits as described below. 

• The Cohort Graduation Rate Policy and Technical Manual (31 pages) - Guidance on the 
calculation of the cohort graduation rates. 

• The Dropout Rate and Exit Adjustment Policy and Technical Manual (24 pages) - Guidance 
on the calculation of the NCES dropout rate and the interpretation and use of the Exit 
Adjustment Window. 

• Multiple smaller guidance documents on specific topics, including the reporting of students 
who remain enrolled after meeting diploma requirements, the use of a specialized type of 
ADM record to add missed outcome data that should have been reported in prior years, and 
policy on assigning cohort years in unusual circumstances . 

Manuals and trainings are revised each year to reflect the most up-to-date information as policy and 
reporting changes are implemented, and to address prior areas of confusion in greater detail. 
Information Technology staff produce and maintain additional technical documentation, including 
the Consolidated Collections File Formats . They are also responsible for running the Data 
(Collection Committee, a meeting of stakeholders from districts all across the state to announce, 
discuss, and get feedback on data collection and reporting changes. The committee meets 10 times 
per year via webinar, and an additional two times in person. 
































Data Collection 

The main sources of data for graduation and dropout reports are the Cumulative Average Daily 
Membership (ADM) collections. These collections include enrollment and outcome data for every 
student who was enrolled at any point during the school year. It also provides information regarding 
events such as graduation, transfers, and dropouts. ODE analyzes data from this collection over 
multiple years in order to create and adjust the cohorts for each school and district, and to determine 
the number of graduates. 

Schools and school districts report enrollment and outcome data to ODE four times each year 
through the Cumulative ADM collections. Each collection includes data from the beginning of the 
school year (July 1) through a specified point (October 1, December 31, May 1, and June 30, 
respectively). By collecting this data on a rolling basis, it enables ODE to flag some issues, such as 
demographics and enrolled grades, early in the process, and to compare each period’s reporting 
against what was reported in a prior period, for consistency and completeness. 

Cumulative ADM data is submitted to ODE by the districts, and has to pass 119 basic quality 
checks during the submission process. These checks are performed automatically by our system to 
catch obvious mistakes, such as missing data or inconsistent data values. Examples of these checks, 
known as business mles, include: 

• The number of days present and days absent may not exceed the number of weekdays 
between the beginning and end of the student’s enrollment. 

• State school funding may not be claimed for students who exceed age limits. 

• The beginning of the student’s enrollment must be reported with an earlier date than the 
end of a student’s enrollment. 

• Diplomas may not be awarded on a date prior to the date one or more of the essential skills 
requirements were met. 

These business rules are designed to catch situations where the data is obviously incorrect. Data 
quality check that flags probable errors rather than certain errors is performed during audit review 
instead. Additionally, highly complex checks that require significant processing time are also 
reserved for the audit window. More information about the audit review process is available below. 

Additional data sources for the cohort graduation rate calculation include: 

• the most recent First Period Cumulative ADM collection; 

• GED and Adult High School Diploma recipient data from the Higher Education 
Coordinating Commission; 

• the Secure Student collection; 

• Spring Membership; 

• the Oregon Migrant Student Information System ( OMSIS) ; 

• the December and June Special Education Collections; and 

• the Limited English Proficient collections. 

ODE provides live and recorded webinar training to schools and districts before the opening of any 
of the above data collections, which offers school and school district staff an opportunity to learn 
about the collections and ask questions. Additionally, ODE published updated collection manuals 
at a minimum of once per year. 



Audit Review 

Once data collections have been submitted, ODE staff conduct additional reviews of the data to 
help ensure accurate reporting by the district. Staff examine the data and look for likely errors and 
unusual patterns in the submitted data, such as districts that did not report any graduates, or districts 
that reported significant changes in enrollment. These records are flagged for review. Districts are 
then presented with the flagged records and an explanation of the potential data quality issue, and 
asked to either edit the record or confirm that it is accurate. ODE calls this system the audit review 
process . 

Many of these audits were aimed at resolving state school funding claims, class size underreporting, 
and other accountability reporting issues, but a significant number target graduation and dropout 
reporting concerns, such as subgroups, outcomes, enrolled institutions, or essential skills concerns. 

A few examples of these audits are: 

• Diploma reported with Math Essential Skill met via OAKS or Smarter Balanced, but ODE 
has no record of an appropriate score. 

• Enrollment for this student was reported up to June 3, 2015 after the diploma reported as 
being granted on Jan 30. 2015. 

• Ethnicity calculated as "Asian, Not Hispanic" in this attending district last period, but 
"Hispanic/Latino" in this period. Please verify that "Hispanic/Latino" is correct, or change 
the ethnicity/race flags as needed. 

• Student was reported as a transfer to another Oregon District, but no further enrollment has 
been reported in this period. There were 110 weekdays between the reported end of the 
enrollment and the end of the period. Please check that you have documentation of the 
transfer. 

These record-level audits are part of the first steps ODE takes to ensure the quality of our data 
collections. ODE’s Accountability & Reporting staff continuously review and improve the way 
records are selected for audits, in order to increase clarity, catch more mistakes, and reduce the 
incidence of false positives. Staff also monitor audit responses and follow up as necessary. 

As an example, in the 2014-15 Annual ADM collection almost 70,000 out of the 700,000 records 
were flagged for audit. By the end of the audit review window, 99% of the Annual Cumulative ADM 
2014-15 audits had been addressed by the school or school district that submitted the record. This 
work by both ODE and district staff significantly improves data quality. 

Exit Adjustment 

Cohort calculations include data from multiple school years, and some of this data is in collections 
that are several years old and cannot be edited in our system. During the intervening years districts 
sometimes learn additional information on students and their outcomes (GED, transfer to another 
state, etc.) that are not captured in existing data. In particular, once districts know who returns to 
school each fall, they need an opportunity to revise reported outcomes, as needed, from the 
previous school year. 

First Period Cumulative ADM includes enrollment records between July 1 and October 1 of the 
school year. Data collection for First Period Cumulative ADM is completed at the end of October, 
after which the data undergoes an audit review process similar to what was described above. 




During the same rime period, ODE conducts the Annual Cumulative ADM Exit Adjustment 
Window. Exit Adjustment is a six-week window which runs from mid-October through the end of 
November. A student’s last Annual ADM enrollment is compared to their earliest First Period 
ADM record (if any), and districts are notified if the Annual ADM outcome code appears to be 
incorrect. This gives the district an opportunity to review their data and make corrections to the 
outcomes or reported enrollment as necessary. 

For example, if a student was reported in Annual ADM as expecting to re-enroll for the next year, 
but there is no enrollment in First Period ADM, the record might receive a suggested code of 
dropout. Conversely, a student who was reported as a dropout in Annual ADM, but returned in 
First Period, might receive a code indicating that they re-enrolled, and so should not have been 
reported as a dropout. 

Exit Adjustment’s suggested codes are updated nightly in order to provide users with ongoing 
feedback about a student’s status as they and other districts add and edit records. ODE provides 
live and recorded webinar training on the Exit Adjustment window prior to its opening each year, 
including guidance on how to interpret and respond to the codes shown, and an explanation of how 
the data will be used to produce graduation/dropout reports. ODE also issues an Exit Adjustment 
Manual each year. 

Cohort Year Appeals 

A portion of cohort graduation rate processing occurs year-round, as high school entry years (also 
known as cohort years) are set, reviewed, appealed, and overridden. In accordance with federal 
guidance, once a student has been assigned a cohort year, it is locked in our data system, and can 
only be changed by ODE staff. Most students do not require any changes to this field once it has 
been set for them, but each year between 300 and 1,800 students do. This can occur because of data 
entry mistakes made by district staff, or because new documentation has been discovered by a 
school district which changes the cohort year that is appropriate for the student. 

Districts complete a short appeal form with information about the student and the reason the 
cohort year is changes. ODE staff review each appeal, including the district’s evidence, if any, and 
existing ODE records on the student, if any, and make a determination to approve the appeal, deny 
it, or request additional information. ODE staff are also responsible for proactively seeking out 
potentially erroneous cohort years by comparing assigned cohort years to existing enrollment 
records and contacting districts for verification. ODE staff conduct numerous other checks 
periodically throughout the year, such as reviewing students who were reported as transfers to 
another district to determine whether the student was erroneously issued a new ID number by their 
new district, and assisting in linking those IDs together to avoid double-counting the student. 

Processing 

ODE’s annual graduation and dropout reporting process begins internally in August or September 
of each year. Changes to any of the data collections used in the rates, policy changes to the way 
graduation rates are calculated, the availability of new data sources, or usability improvements may 
require ODE to updates its calculation methodology. ODE staff review these changes and consult 
with the external Accountability Reporting Advisory Committee (ARAC), a stakeholder group 
composed of a representative sampling of school district staff, to evaluate areas of concern and 
proposed changes to the process. 



IT staff and Research Analysts then work together to evaluate, develop, and implement any 
programming changes that are needed for the coming year. At this time. Research Analysts also 
work to gather supplemental data sets from other agencies, such as lists of students who were 
awarded a GED, which are later incorporated into the reports. 

Next, institutions data is analyzed to determine which of Oregon’s approximately 1,500 schools and 
districts were degree-granting institutions in each of the relevant years, and therefore which schools 
are accountable for which students’ outcomes. Appropriate adjustments must be made for schools 
that closed, reopened, added or removed grade levels, merged, relocated, or had major changes to 
their enrollment in any of the five years being considered. 

Once any necessary programming changes have been made and thoroughly tested, test versions of 
the graduation rates are generated, including student-level records for approximately 100,000 
students per year. Accountability & Reporting Research Analysts review the results to make sure 
that every field is calculated correctly, creating duplicate versions of the official reports in order to 
check for inconsistencies. The process considers up to 5 years of records from up to 9 different 
collections, so ensuring that every student is accounted for correctly, with appropriate subgroup 
identifications and enrollment and outcome data, is a complex and critical task. 

Resolving outcomes alone requires processing over half a million records each year. A set of mles 
and hierarchies is used to resolve a student’s many records into one final outcome. A separate set of 
mles is used to assign the student to one of the accountable institutions selected in the previous step, 
and a third set of mles is then applied to identify a student’s gender and ethnicity, and which of 
seven subgroups the student should be included in. Details of these rules are published in the 
Cohort Graduation Rate Policy and Technical Manual, and are covered in the yearly trainings. 

Historically, running the complete process took so long that it had to be started at the close of 
business so that it could run overnight. Due to IT and Accountability & Reporting improvements 
to the system, including reducing the number of corrections that had to be handled manually and 
streamlining the data submission system, processing can now usually be completed in 4-6 hours. 

Student-level data quality checks begin at this point, including analysis of students whose outcomes 
have changed from year to year, students whose last outcome was an unconfirmed transfer event, 
students whose last event indicated that continued enrollment should be expected, and students who 
are counted as dropouts but might have earned a credential in a prior year. ODE staff review each 
student’s records and contact districts as necessary when ODE determines that a reporting error 
may have been made. ODE staff also review student records for each override applied in the 
previous year to determine which ones should be carried forward for the current year’s processing. 
Additional student-level and aggregate validation checks continue throughout the validation window. 

District Validation of Data 

Once staff are satisfied that the processing is complete and accurate, and any necessary corrections 
have been implemented, the data are moved to the Achievement Data Insight (ADI) application, 
which allows authorized staff in schools and districts to see them. Historically, districts were only 
able to view their aggregate data, and had to request a file of student-level outcomes from ODE 
Research Analysts, but in 2012-13, ODE’s IT staff created the ADI application. The application is a 
tremendous improvement over the previous system, allowing districts to review all of their data in 




greater depth, as well as including several other user-friendly features such as sorting, filtering, asking 
questions through the application, downloading the data to a spreadsheet, and accessing data from 
previous years. 

The improved application automates several previously manual functions, provides authorized 
district staff secure access to detailed student-level data, allows ODE staff to monitor activity on the 
validation in order to create customized reminder and notification systems for districts that have not 
reviewed their data, and frees ODE staff to devote more time to data quality measures. 

Training on the use of the ADI application is conducted during the summer, but is also repeated as 
part of the larger graduation and dropout validation training, conducted shortly before the validation 
window opens. As with other trainings, these are conducted live and recorded for later review, and 
accompanied by updated technical manuals with details on the process. 

Graduation and Dropout data are available in the ADI for about 6 weeks, beginning at the end of 
October. School and school district staff can view and download early versions of the aggregate 
data that will eventually make up the published reports, as well as detailed student-level data. Two 
ODE staff members in Salem, as well as three contractors (ODE’s regional ESP data collection 
support partners) across the state, are available to answer questions and assist districts in making 
corrections to their data. ODE staff members also track which institutions have reviewed their data, 
and send reminders to those who have not. In the most recent validation, the application logged 
over 4,300 views of the validations. 

Districts with questions about student outcomes contact an ODE Research Analyst, using ODE- 
issued secure student IDs to clearly and specifically identify the student without disclosing personally 
identifiable information in an unsecure manner. ODE staff review all records for the student, and 
explain how the data in the student’s resolved record was determined. Generally speaking, staff then 
direct the district in the appropriate way to edit the underlying data sets in order to add a missing 
outcome. In some cases, ODE staff must introduce a manual override to the student’s record, in 
order to correct data. 

In the process of calculating graduation rates for 2011-12, ODE staff added more than 4,800 manual 
updates and overrides to the system. Since then, improved data quality checks throughout the cycle 
of the collections, as well as improvements to the mechanisms for collecting and updating data, have 
reduced this number to fewer than 100 updates and overrides for the 2014-15 graduation rates. 

Throughout the validation window, ODE staff review data and contact districts to confirm unusual 
values or assist in making corrections. This includes districts with unusual increases or decreases 
from the previous year’s rates, districts with exceptionally high transfer-out rates, and many other 
validations. Districts are contacted on an individual basis, and may also sign up for general 
reminders via a weekly newsletter . 

This year, in addition to the usual checks ODE performs, ODE contacted by email every district 
whose rate was below the state average, as directed by the Budget Note. Because state-level rates 
typically increase over the validation period, and the contact was made prior to the rates being 
finalized in order to allow districts time to edit, an email was also sent to every school district whose 
preliminary rate was up to 1 percentage point above the state average in order to ensure that all 
districts with rates below state averages were contact. The email informed the district what their 





currently calculated graduation rate was and asked them to confirm that it is accurate, as well as 
providing links to training materials and policy and technical manuals. 

The email was distributed in early December, approximately 1 week before the end of the validation 
period, to 84 school districts. At the time of the email, ODE records indicate that 60 of the 
identified districts were already in the process of or had completed reviewing and correcting their 
rates. 

Over the course of the validation window, Oregon’s statewide graduation rate increased from a 
preliminary 71.2% when the validation first opened, to a final 73.8% at the end of the window, a 
change that was the result of 800 additional transfer-out events and 600 additional graduation events 
added during the validation period. Half of a percentage point of that increase occurred after the 
email specified in the budget note was distributed, which is the typical size of the increase during 
those last two weeks in previous years. 

After the validation closes in mid-December, ODE staff review and apply any final changes. Most 
necessary corrections can be made by districts themselves, via the previously described collections 
and some enhancements made to ODE’s data collection process in the last few years, but some 
changes still require ODE staff to review and implement manual overrides on a student-by-student 
basis. Once this process is complete, a final refresh of the data is generated in early January and 
displayed to schools and school districts through the ADI. Districts have a short final opportunity 
to review the data and notify ODE of problems before the reports are published at the end of 
January. 

Conclusion 

As described above, ODE conducts and extensive training process for districts so that they can 
submit accurate data for use in cohort graduation rate calculations. ODE also provides extensive 
opportunities for district staff to review and edit their data prior to the publication of these rates. In 
the most recent year ODE, as directed, also contacted all districts with rates below the state average. 

Despite the extensive system that is in place, ODE works each and every year to improve and 
streamline this process, and to improve quality of the data used to calculate cohort graduation rates. 
We work with stakeholders on a regular basis to review and revise our calculation rules and to 
improve the efficiency and accuracy of data reporting. We look forward to working with the 
legislature and other groups as we continue to improve our data collection and processing systems in 
the years ahead 



Appendix A: Validation Interface Examples 


Student-level Data 

Actual display is a single long table; columns have been condensed and personally identifiable information removed. 


Validation - 4-Year Cohort Graduation Rate 2014-2015 


Reporting Year: j 2015-2016 


▼ I Click here for instructions 


The four-year cohort graduation rate is the percentage of students in a cohort, adjusted for transfers into and out of the school, district, or state, that graduate with a regular high school diploma within four years of entering high school. A cohort is 
composed of students who first started high school in a given school year. 


i you are currently logged in for \ with Institution ID 

Data Owner: Isabella Jacoby (503) 947 - 5878 


Validation Open: 10/29/2015 

Validation Closed: 12/11/2015 CloS@d 

Public Release: 01/28/2016 ^ U 
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Appendix B: Communication Examples 


rom: 

3 : 

c 

ubject: 


0 JACOBY Isabella 


SSID 


Sent Tue 12/1/2015 10:2 


You may want to review records for SSID . This student is currently coded as a dropout, but 

ADM records suggest that they may have previously earned credential, in which case they would 
qualify for a 6B ADM End Date Code. If you determine that they student does have a prior credential, 
please contac> to request to have the Annual 

ADM collection reopened so that you can edit their record. 


JACOBY Isabela 


hort Graduation Rate Validation 


Thanks, 

Isabella 


Isabella Jacoby 

Research Analyst - Data Collection Lead 

Offce of Learning | Instruction, Standards, Assessment & Accountability Unit 
Oregon Department of Education 
503-947-5878 | isabslk-iacobv g stats.orus 


Sent Thu 12/3/2015 10:1 


E's budget notes this year, the legislature has asked us to contact a number of districts, 
rs, to confirm that your cohort graduation rate is accurate. If this message Is more appropriate 
else in your district, please forward to them and let me know. 

four-year cohort graduation rate is currently • ■ I see that your district has already 

four-year cohort graduation rate in the Achievement Data insight (ADI) application, so if this 
te, no further action is necessary. If you have not already viewed them, you may also wish to 
re-year cohort graduation rate and NCES dropout rate validations, which are also open at this 
re their own validation tiles in ADI. The Achievement Data insight validations will refresh today 
play updated data. 


Please remember to view the graduation and dropout validations for this year before they close. There are three 
validations: 4-year Cohort Graduation, 5-year Cohort Graduation, and the NCES Dropout validation. Our records 
currently indicate that none of them have been viewed by your district. If this email would be more appropriate for 
another staff member, please forward to them and let me know. It appears that the dropout and 4-year cohort 
graduation rates have changed significantly from last year's values for your district, so please ensure that they are 
accurate. 

You can view the details and the student level data underlying the rates by logging into the Achievement Data insight 
(ADI) application. Training on the use of the application, and on the validations in general, was conducted October 13, 
2015, and a recording is posted at https://district.ode.state.or.us/search/page/?id=250 . Please note that all corrections 
must be made at the student level, typically by editing your district's Cumulative ADM submission. If you are not sure 
how to correct a student's outcome, please contact your ESP partners or me with the student's SSID and the type and 
date of the outcome they should have. 

Details of ODE's calculation methodology are available at 

http://www.ode.state.or.us/wma/data/schoolanddistrirt/5tudents/docs/cohort-graduation-rate-policv- 

manual 201415.pdf and 

http://www.ode-state.or.us/wma/data/schoolanddistrict/students/docs/dropout exitadi manual 20142015.pdf . The 

deadline for corrections to the graduation and dropout data is Friday, December 11,2015. 

Thank you, 


if the rate is inaccurate, or you wish to view more information about how we calculated this rate, you can 
view the details and student level data underlying this rate by logging into the Achievement Data Insight 
application. Training on the use of the application, and on the validations in general, was conducted 
October 13, 2015, and a recording is posted at https://district.ode.state.or.us/search/page/2id-250 . Please 
note that all corrections must be made at the student level, typically by editing your district's Cumulative 
ADM submission. If you are not sure how to correct a student's outcome, please contact me with the 
student's SSID and the type and date of the outcome they should have. I know that many districts are 
already in the process of correcting this data, and we appreciate all of your hard work. 

Details of ODE's calculation methodology are available at 

http://www.ode.state.or.us/wma/data/schoolanddistrict/students/docs/cohort-graduation-rate-policv- 

manual 201415.pdf . The deadline for corrections to the graduation and dropout data is Friday, December 
11, 201S. 

Thank you, 

Isabella Jacoby 


Isabella Jacoby 

Isabella Jacoby 

Re March An»iyst - Data Collection Lead 

OWc« of learning | Instruction, Standards. Assessment & Accountability Unit 
Orefo" Department of Education 
503-947-5878 | tubfnufcqpv.g jms.gt.u? 


Isabella Jacoby 

Research Analyst • Data Collection Lead 

Off ce of Learning | Instruction, Standards, Assessment & Accountability Unit 
Oregon Department of Education 
503-947-5878 | isabella lacobv gstate or us 


























Sent Tuesday. December 01.201511:09 AM 
Subject 


The student has ADM records ndicatng enrolment from September 2014 to May 2015 m 3' s penod Cumulative ADM 14-15. If 
that record is accurate, you will want to make sure that the student's enrollment is included in Annual ADM 14-15 as wel, so that 
you wii get appropriate state school funding, and you can put the awarded diploma on that record. 

if the 3' s penod enrollment is not accurate, and the student has no? been enrolled dunng the 2014-15 school year, then yes: use 
a program type 14 record to report the dploma without daim>ng ADM fundng. 

Thanks. 

Isabela 


From: 

Sent: Tuesday. December 01. 2015 11:03 AM 
Subject: 

Isabela. 

i am prepanngto re-upload our program type code 14 records, is that how I should report the outcome for this student? If so, 
HI add her to the fie first. I tried searching for her in the 14-15 cum ADM, and she's not there smce she was last enroled m 13* 
14. 


From:.ACOBYIsabella [ma :o:sabe a.iacoovfflstate.or.usl 

Sent: Tuesday. December 01. 20151052 AM 

Subject: 


0 See more about JACOBY Isabella. 


TTTT 


rruaic 

Sent Tuesday, December 01, 2015 10:48 AM 

To: 

Subject: 

Isabella. 

According to .this student received a diploma on 



From: JAC06Y Isabela 

Sent: Monday. November 16. 2015 12:09 PM 

To: 

Subject: 


Please review your records for SSID ADM records indicate that the student met diploma requirements m June 2014, an 

was not awarded a diploma at that time (4F ADM End Date Code). I see that the student was reported as re-enroSng in 2014- 
15, but only in 1“ 2"*, and 3' : periods. I aso see that you confirmed your audit on this student d l n penod. but this is not an 
appropriate response to this type of audit. 

If the student dd re-enroll, pease make sure that all of ther 2014-15 enrolment is nduded in Annual ADM 14-15, and that the> 
appear n l r penod 15-16 if appkabte. If the student did not re-enroll, or is not currently enroAed with you, you must award tht 
student their diploma and report the award of the dptoma in Annual ADM 14-15 |4A ADM End Date Code). If the student neve 
met their diploma requirements, and the 4F code was reported incorrectly, please let me know so that I can adjust the 
student's final outcome. 


Isabella 























Appendix C: Training Examples 


This section contains only a few selected slides from a few of ODE's trainings. Trainings can easily run more than 80 slides each, and last up to 90 minutes. 


First Period Cumulative ADM Trainings 

One training for new submitters, which covers how data is submitted and validated; second training for all submitters, which covers the collection rules and 
definitions 


Submitting the Data 



Errors & Audits 



Errors 

What are Errors 


Records in the error queue will not be part of the data 
collection until the error has been corrected. 


Errors are generated by specific rules in the application. They appear in your 
Review Errors Menu. Errors are always wrong and must be corrected (unlike 
audits). 

After you have submitted the data, ODE’s system processes the record/s. Those 
without errors are entered into the collection. 


Example: Student already reported for same time period in another school/district 


Example: Enrollment Date is later than the End Date 


If you are not sure why you 
have received an error, you should 
contact Beth Blumenstein or your ESD 
partner with the student's SSID and 
the error message you’re seeing. 


Example: Required data fields are left blank 














































Submitting the Data 


WHAT 

Student Demographics 


Process & 
Content Manual 


Subgroup Flags: 

District Special Education 

Student was enrolled in a district special education program during the 
school year and received general education classroom instruction for 
LESS THAN 40% of the time f Federal Placement Codes 32 - 41 ) . 

including non-regular education settings. 


Note: 

Special Education flag 
must be Y when the 
District Special 
Education Flag is Y. 
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Submitting the Data 


WHAT 

Student Outcomes 


Enrollment End Date Code 4 A & B: Completed Program 


Code 

Name 

Description 

4A 

Completed Diploma-Track 
Program, Met Requirements 
for HS Diploma, and was 
awarded a HS Diploma 

Students completed an approved program of study and met all 
state or district requirements for a high school diploma, and 
were awarded a high school diploma. 

4 B 

Completed Diploma-Track 
Program and Did Not Meet 
Requirements for HS 

Diploma 

Students completed an approved program of study for high 
school completion, but did not meet all state or district 
requirements for a diploma 


Diploma Type Codes: 

1 = Regular 

2 = Modified 

3 = Adult 

4 = Extended 



Note: Modified 
Diplomas are reported 
with an End Date Code 
4A and Diploma Type 
2, not an End Date 
Code 4B 


Submitting the Data 


WHAT 

Instructional Info 


Program Type: 14 


14: Report any additional outcomes (credentials, etc.) that 
were not previously reported. A credential could include a 
GED or other certification other than a regular high school 
diploma. Transfer and deceased codes are also allowed 
under this program type. Reported with o days 
present/absent, o hours of instruction, and o FTE. Tuition 
Type Code is P, and no funding is received for these 
records. 


These records must be resubmitted in 2 nd , 3 rd , and Annual ADM Collections. 


Program Type 14 vs Exit Adjustment: 

• 14 = Adding information 

• Exit Adjustment = Changing information 
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Production Download 



The production download contains the data that have been submitted to 
the collection. It will not show records that have unresolved errors in the 
error queue or records that are still being processed. 


ALWAYS request a production download when you are done submitting your 
data and fixing your errors. Then, CHECK YOUR DATA, comparing what you 
should have submitted and what ODE has received. 


When collection is open: 



When collection is closed: 


Status Tracking 




Class Roster 




Daily Membership (CumADM) 


SSID (Secure Student ID) System 













































































Graduation, Dropout, and Exit Adjustment Trainings 

One training for exit adjustment and data submission; second training for calculations and validation 





How do I see the Calculated End 
Date Code? 


Speaking Assessment of Essential Skill Code 

Speaking Assessment of Essential Skill Administration Date 
Mathematics Assessment of Essential Skill Code 

Mathematics Assessment of Essential Skill Administration Date 


District Special Education Program Flag ■ a 


Attending School Full Academic Year (FAY) Flag YlJ N 
Attending District Full Academic Year (FAY) Flai 


Calculated Average Daily Membership (ADM) Amount 0000 


Activity Date: 10/8/2012 2:28:22 PM 


Preferred Name First | 
Legal Name First | 


Attending District ID | 

Resident County 


What does the Calculated End Date 
Code mean? 

o The calculated end date code is a mechanism for ODE 
to communicate with districts regarding students who 
may have been reported incorrectly, and to provide 
additional information about students to districts. 

To be considered in calculated end date code processing: 

O ADM Enroll Date 7/1/2014 or later for Annual ADM 2014-15 Records 

O ADM Enroll Date 7/1/2015 or later for 1 st Period ADM 2015-16 Records 


New this year: The process will take ADM Diploma Type codes 
account. 


into 
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The most common calculated end 
date codes: 

O 1A, ID, 2A, 2D, 2?: The student was reported in First Period 
Cumulative ADM 15-16 

o 2A if the student was reported in 1 st period as attending an 
accountable district, or an ODE-sponsored charter 
o 2D if the student was reported in 1 st period as attending JDEP 
o 2? if the student was reported in 1 st period somewhere else 

O 3E, 3F: The student was last enrolled after May 15, 2015 (for 
3F) or before May 15, 2015 (for 3E), and did not either re¬ 
enroll in 1 st Period Cumulative ADM or report a different 
outcome in Annual (transfer out, graduate, specific dropout 
code...) 

See the Dropout and Exit Adjustment Manual for the complete list 


26 10/13/2015 


Differences between Reports 



NCES 

4-yr Cohort 

5-yr Cohort 

Denominator 

Dropout Rate: Fall 

Membership 2014-15 in 

Grades 9-12 

Grad Rate: Dropouts + 
Graduates 

Adjusted Cohort 
(students with 2011-12 
cohort, minus students 
whose last event was 
deceased or transferred 

out 

Adjusted Cohort 
(students with 2010-11 
cohort, minus students 
whose last event was 
deceased or transferred 

out 

Graduates 

Regular or Modified Diploma 
awarded between Sept 1, 

2014 and Aug 31, 2015 

Regular or Modified 
diploma earned by 
students in the adjusted 
cohort between July 1, 
2011 and Aug 31, 2015 

Regular or Modified 
diploma earned by 
students in the adjusted 
cohort between July 1, 
2010 and Aug 31, 2015 

Dropouts 

Students reported as school- 
year dropouts between July 1, 
2014 and June 30, 2015, plus 
students reported as 3F 
between May 1 and June 30, 
2014. 

No cohort dropout rate 

No cohort dropout rate 
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ADM Program Type 14 

o These records can be used to submit diplomas, other 
credentials, and some non-credential outcomes, both 
for the current year and for prior school years. 


Be careful when adding these records - you must use the 
actual event date in most cases. Using a date that is too 
late or too early may cause the record to be incorrectly 
excluded from this year's calculations. 


33 10/13/2015 


Appealing Cohort Years - Review 


o Cohort years may be reset because: 
o A data entry error was made 
o New documentation is discovered 


o Cohort years may NOT be reset solely because: 
o A student skips a grade 


o A student repeats a grade or is demoted 

o A student moves from one district to another 

o A student spends a year or more not enrolled in school 




Your ESD partners ( www.ode.state.or.us/go/AssessmentHelp ) can help 
you appeal a currently assigned cohort year. See 
http://www.ode.state.or.us/news/announcements/announcement.asp 

x?id=8564&typeid=4 for instructions. 
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Click column headers 
to sort 


Column Headers 


J^^)ownloatn)at^^^^^Hide^ilte^ 
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Accountable Institutions - 
Examples 

o Student enrolls in Magenta SD, transfers to Orange SD in September 2012, 
and graduates from Orange SD in June 2015. Student is accountable to 
Orange SD. 


o Student enrolls in Aqua SD, at Daisy Johnson Charter School and remains 
enrolled there through June 2015 without earning a credential. In 
September 2015, the student transfers to Chartreuse SD. Student is 
accountable to Aqua SD and Daisy Johnson Charter School. 


o Student enrolls in Crimson SD, at Coulson Memorial HS. Student transfers 
into a Crimson SD district-level program in May 2014, and remains there 
until withdrawing from the school in June 2015. Student is accountable 
to Crimson SD and to Coulson Memorial HS. 


o Student enrolls in Crimson SD, at Coulson Memorial HS. Student transfers 
to Aubergine SD, then returns to a district-level program at Crimson SD in 
May 2014, and remains there until withdrawing from the school in June 
2015. Student is accountable to Crimson SD, but not to any school within 
Crimson SD. 


39 


Clarifying Similar Outcomes 

o Transferring to another institution (Documented outcomes only) 

o If the institution is another Oregon School District use 2A 
o If the institution is a public program that does not offer a high school 
diploma, then use 2D 

o If the institution offers a program leading to a high school diploma 
(including private or home school and some Job Corps), or the student 
has moved to another country, then use 2B or 2C 
o If the institution offers a program leading to a bachelor's degree, then 
use 5D (not including expanded options or other college credit programs 
for students still enrolled in high school) 
o If there is no secondary education program, then use 3C or 3D 
o Transferring to a non-traditional setting that is an off-campus program 
offering or an extension of the district, that is tracked by district, then use 
IB or 1C; 

o Transferring to a non-traditional setting (residential facility, correctional 
institution, community or technical college program) not approved or 
tracked by the district, then use 3D 


55 10/13/2015 


Example 

o A student attends Carol Danvers High School from 

September 2010 to May 2014. The student meets diploma 
requirements, but elects to remain enrolled for a fifth year, 
so she is reported with a 4F ADM End Date Code, 
o In September 2014, the student enrolls in 

for her fifth year of high school (note: 
both schools are in the same district). She completes the 
year there, and in June 2015 is awarded her diploma and 
reported with a 4A ADM End Date Code. 


In the 2014-15 Cohort Graduation Rates, the student's 
outcome will be High School Diploma Awarded, and her 
Accountable School will be Carol Danvers High School. 




































































